Secrets of Atari hard-disk partitions

By Al Fasoldt

Copyright (c) 1994 by Al Fasoldt. All rights reserved.


When Atari laid out the way hard disks should operate with the ST in
1985, it choose to stick with the method already in use for PCs at
that time. This was a wise choice. The PC disk format was widely
understood and easy to work with. PC standards have changed in the
ensuing decade, and so have the standards used by Atari computers.
But the basic operation of a hard drive attached to an ST, TT, STacy
or Falcon should present no problem to anyone familiar with servicing
or setting up PC hard disks.

All hard disks that work with Atari's so-called 16/32-bit computers
(or "Sixteen/Thirtytwo" -- "ST") use 512-byte sectors. This is true no
matter how large the hard disk is. Let's back up a little and take a
look at the way disks store information.

To make disk storage fast and efficient, files of any size are
divided into chunks called clusters when they are stored on disk. An
ASCII text file consisting only of the letter "A" is one byte long in
your computer's word processor, but it takes up much more space than
that when it is stored on disk. Because the smallest storage space on
a hard disk is a cluster, that single-byte file occupies an entire
cluster on the disk. (The "A" would appear at the start of the
cluster, but the rest of the bytes in the cluster would be so-called
null characters -- characters that don't mean anything.)

How big is a cluster? Normally, a cluster is two sectors long. So
that means every file takes up at least 1,024 bytes (512 bytes X 2).
This gets a little complicated when large drives are used. An
explanation of how a single disk drive can appear to be many drives
should help.

For a number of technical reasons, a hard drive often works better
when the computer deals with it in sections. These are called
partitions. All computers of any kind make use of this technique. The
drive is partitioned into two or more semi-independent areas that the
computer sees as separate drives. (The computer has no way of knowing
if Drive C: and Drive D: are physically separate or not.)

One advantage of partitioning a drive is found in the way the
computer deals with sectors. It's just plain impossible for a modern
Atari to handle more than a finite number of sectors on each drive.
(Don't blame Atari for this. Every computer has a limit.) For older
Ataris using versions of The Operating System (TOS) prior to 1.04,
the sector limit is 32,768. Newer models, with later TOS
versions, have a sector limit of 65,536. If you do some simple
mathematics, you'll see how this limit comes into action: No matter
how large the hard disk is, the computer will only be able to work
with either 16 megabytes or 32 megabytes, depending on the TOS
version. (Here's the math: A limit of 32,768 sectors multiplied by
512 bytes per sector equals 16,777,216 bytes. That's exactly 16
megabytes the way computers count, and the other calculation yields
exactly 32 megabytes.)

What's this mean? To put it simply, TOS forces all of us to use
partitioned hard drives if our hard disks are larger than a certain
size. Actually, I'm not telling the whole story. There is a way
(which I will explain shortly) to fool TOS into letting partitions
get a lot bigger, but they can't be any size; there is still a limit.

So if you're planning on replacing a small hard disk with a larger
one -- one that holds 120 megabytes, for example -- you'll have to
make a choice: Partition it into at least three separate drives or
fool TOS into thinking the drive has no more than 65,536 sectors
(taking it for granted that you are using a modern version of TOS).

If you stick with the normal TOS method, you'll be using GEM
partitions. These have 512 bytes per sector and two sectors per
cluster. (GEM stands for Graphics Environment Manager, which has
nothing to do with a hard disk, but that's the name these partitions
were given.)

If you want to make larger partitions -- ones that are bigger than
GEM sizes -- you'll need to choose BGM ("Big GEM") partitions. These
can contain many more sectors, but the actual sector count is not
disclosed to TOS. As far as TOS knows, the partition ALWAYS has
65,536 sectors or less. As in GEM partitions, clusters are two
sectors long.

TOS is kept in the dark by a technique that uses logical sectors.
Each logical sector is made up of many physical sectors. What TOS is
told is the number of logical sectors, not physical sectors. The
scheme works quite well. Some very old programs don't run properly
when your drive uses these larger BGM partitions, but most modern
programs have no problem with them.

BGM partitions can be as large as 512 megabytes. This is wonderful,
right? Well, not really. While it's great to be able to hang a
512-megabyte drive onto your computer and treat the entire disk as
Drive C:, you sacrifice two things. One is generally acknowledged and
the other is not.

The first sacrifice is efficiency. In order to get TOS to go along
with a 512-megabyte partition, the hard-drive software bunches up
eight sectors and reports them as a single sector to TOS. Two of those
large logical sectors are then stuck together into a cluster.
Remember, the smallest unit of disk storage is a cluster. So that
cluster contains 16 sectors. Doing some math, we can see that the
smallest unit of storage on a 512-megabyte partition is 16,384 bytes
(512 bytes X 16 X 2). This means every file, no matter how small, will
take up at least 16,384 bytes of space on the drive.

That probably seems like a lot of wasted space for all the small
files on your drive. But it's even worse than it seems. EVERY file is
likely to be affected. On average, the last cluster of space for each
file is only half used, so the average amount of wasted space --
called slack space -- is half the size of one cluster. On a
512-megabyte partition, this slack space amounts to 8 kilobytes per
file. If the drive holds 1,000 files, the total slack space is 8
megabytes.

Folders are a special case. They are actually files that contain
references to other files. Because they are files, they take up one
cluster of space on a drive. The more folders you have, the more
space is wasted on a BGM partition. On my main storage system, I have
more than 1,000 folders, each one of them "empty." (Folders are
always shown as empty in a directory listing.) They take up 16
megabytes of space -- hardly anybody's definition of "empty"!

The second sacrifice is seldom mentioned. One reason for using a much
larger hard disk is to store more files. (This makes sense, right?)
But you cannot, strictly speaking, store more files on a large
partition than you can on a small one.

Say that again? You heard it right. A large disk partition cannot
store more files than a small one because each file takes up at least
one cluster, no matter how large or small the disk is, and the number
of clusters is fixed. On an Atari computer with a modern version of
TOS, no hard disk partition can hold more than 32,768 files. To
accomplish this, each file would have to be no larger than a single
cluster (two sectors). In reality, the limit is a little less than
this, because some of the clusters are reserved for use by the drive
itself, and others must be used for folders. (A partition with no
folders can hold only a few hundred files, because the root directory
is a fixed size.)

This is more trivia than anything else, since no one is likely to try
to store 32,000 1,024-byte files on a disk. But the facts are clear:
A large partition cannot store more files than a small partition if
the files are all small.

Here is how much space a single file takes up on a disk (in other
words, this shows the cluster size), even if the file is only a few
bytes in length, assuming a TOS version of at least 1.04:

GEM partition, maximum size of 32 megabytes: 1,024 bytes.

BGM partition, maximum 64 megabytes: 2,048 bytes.

BGM partition, maximum 128 megabytes: 4,096 bytes.

BGM partition, maximum 256 megabytes: 8,192 bytes.

BGM partition, maximum 512 megabytes: 16,384 bytes.


The lesson here is to weigh the advantages and disadvantages of BGM
partitions before choosing one or the other. BGM partitions make
everything easy: You need to deal with a minimum number of drive
partitions, and disk operations are often faster. (Moving a file from
one folder to another on the same partition is done very quickly,
since all the operating system does is rewrite the pathname from one
entry to another.) But they also waste space. Hard disks are cheaper
than ever, and that wasted space doesn't cost much in dollars. But it
does seem like an unnecessary loss of space otherwise, especially
considering the increasing slack space as partitions grow.


(This document cannot be printed or republished in any form without
the express consent of the author. Al Fasoldt is Systems Editor of
the Syracuse Newspapers in Syracuse, New York, and author of a
syndicated newspaper column on computers and technology. He can be
reached at this Internet address: a.fasoldt@genie.geis.com.)